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Abstract 

The  intended  end  product  of  our  current  research  project  is  an 
"intelligent”  multimedia  tutoring  system  for  procedural  tasks,  and  in 
particular,  for  repair  of  physical  objects.  This  paper  presents  the  data 
structure  that  will  be  used.  It  is  a  graph  with  five  types  of  nodes  (mental, 
abstract,  motoric  or  action,  visual,  and  verbal)  and  two  types  of  links 
(subconcept  and  pointer).  The  graph  examples  given  in  the  paper  are  knowledge 
representations  of  conceptualizations  .that  people  might  have  for  a  simple 
object,  a  flashlight.  We  show  how  the  representations«are  used  for  choosing 
actions,  planning  strategies,  making  inferences,  and  designing  instructions. 
We  give  the  plan  for  computer  implementation  of  the  tutoring  system.  We 
report  previous  applications  of  this  knowledge  representation,  including  how 
it  can  be  derived  from  experimentally  observed  behavior.  And  we  compare  our 
knowledge  representation  to  others. 
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I.  Introduction 

A.  Research  Overview 

Our  current  research  is  focused  on  how  to  design  interactive  multimedia 
instructions  for  procedural,  and  more  specifically,  repair  tasks.  A  repair  is 
defined  as  follows:  Suppose  we  have  an  object  which  has  both  structure  and 
function.  By  repair  we  mean  a  modification  of  the  structure,  when  the  object 
has  stopped  functioning  or  is  functioning  incorrectly,  in  order  to  make  it 
function  properly  again. 

The  main  goal  is  to  find  the  theoretical  and  practical  principles  for 
developing  an  "intelligent"  computerized  system  for  repair,  while  also 
actually  developing  such  a  system. 

The  method  of  the  research,  which  we  have  just  begun,  is  to  build  an 
experimental  implementation  using  the  data  structures  and  procedures  that  act 
on  them  which  are  presented  here  and  which  come  from  our  theoretical  framework 
(Baggett  &  Ehrenfeucht,  1982,  1985).  The  system  will  be  used  in  the  domain  of 
repair  tasks.  The  main  questions  are: 

1.  Is  our  theoretical  representation  of  knowledge  (see  below)  an  adequate 
description  of  the  knowledge  a  person  must  have  to  perform  repair  tasks? 

2.  Can  our  multimedia  knowledge  representation  be  efficiently  implemented  as 
a  tutor  for  repair  tasks? 

3.  What  types  of  tutor/user  interactions  can  be  used  to  communicate 

successfully  the  tutor's  knowledge  to  the  user?  Theoretically,  how  can 
new  information  most  effectively  be  entered  into  an  existing 

representation? 

4.  What  is  the  role  of  modalities  of  information  (moving  video,  still  photos, 

color  graphics,  verbalization)  with  respect  to  the  concepts  people  form, 
and  with  respect  to  their  being  able  to  execute  these  concepts,  i.e., 

perform  the  repairs? 

5.  (Related  to  4.)  What  conceptualization  does  a  person  form  as  a  result  of 

the  task?  How  does  information  from  the  tutor  influence  the  person's 

conceptual i zati on? 

B.  Purpose  of  this  Article 

The  purpose  of  this  paper  is  to  present  concrete  examples  of  the 
multimedia  knowledge  representation  which  has  not  yet  been  implemented  but 
vrfiich  we  plan  to  use  as  a  data  structure  for  the  tutor.  We  take  a  relatively 
simple  example,  a  flashlight.  In  III. A.  below  we  show,  in  fairly  complete 
detail,  the  knowledge  representations  for  two  different  fictitious  human 
conceptualizations  of  the  flashlight,  as  viewed  in  our  framework  (Baggett  « 
Ehrenfeucht,  1982).  In  III.B.  we  show  how  knowledge  representations,  as 
viewed  in  our  framework,  are  formed  and  used  for  selecting  actions,  for 

strategies,  and  for  making  inferences.  III.C.  focuses  on  preparing 

individualized  instructions  for  a  repair  task,  and  III.D.  gives  an  overview  of 
our  plans  to  implement  the  ideas  in  an  "intelligent"  program  which  would  be 
used  as  a  teacher  in  a  repair  task. 


II.  Theoretical  Background 

A.  The  Theoretical  Framework 

The  theoretical  framework  guiding  this  work,  which  has  given  rise  to  the 
explicit  knowledge  representation  and  processes  acting  on  it  presented  below, 
concerns  how  people  process  information.  It  deals  with  concept  formation  and 
withencoding,  retaining,  and  using  information  from  multimedia  stimuli. 
Within  the  framework  a  concept  is  represented  as  a  graph  with  two  kinds  of 
associative  links,  and  nodes  corresponding  to  elements  from  different 
modalities,  e.g.,  motoric  (action),  visual  (pictorial),  linguistic,  and 
abstract.  (Details  are  given  in  Baggett  &  Ehrenfeucht,  1982.)  There  is  thus 
a  single  conceptual  memory.  (Many  other  researchers  also  assume  a  single 
conceptual  memory,  for  example,  Anderson  &  Bower,  1973;  Carbonell  &  Collins, 
1974;  and  Norman  &  Rumelhart,  1975.)  The  memory  is  connected  to  multiple 
processors.  Processors  can  be  thought  of  as  processing  visual  input,  auditory 
input,  tactile  input,  and  so  on.  Processors  take  input  signals  and  put  them 
into  memory,  forming  concepts. 

Concepts  are  not  independent.  If  processor  A  builds  a  concept,  then 
processor  B  can  build  another  concept  that  is  a  part  of  it.  This  gives  the 
hierarchical  structure  and  the  multimedia  aspect:  Visual,  auditory,  and 

motoric  information,  for  example,  can  be  part  of  the  same  concept.  A 
theoretical  approach  similar  to  ours  is  that  of  Jackendorff  (1983),  who  states 
that  there  is  a  single  level  of  mental  representation  at  which  linguistic, 
sensory,  and  motor  information  are  compatible. 

We  plan  to  analyze  specific  hypotheses  arising  in  this  new  project  within 
our  framework,  and,  importantly,  we  expect  that  there  will  not  be  a  situation 
in  which  we  have  to  modify  the  framework  (e.g.,  add  a  new  link  type)  in  order 
to  do  the  analysis.  The  basic  data  structure  does  not  seem  to  be  too 
restrictive,  and  it  appears  that  its  efficiency  is  as  good  as  any  other 
proposed. 

B.  Selecting  a  Knowledge  Representation 

Work  on  knowledge  representation  in  artificial  intelligence  has  resulted 
in  a  variety  of  types  of  data  structures  for  storing  information  in  computer 
programs,  and  in  procedures  that  manipulate  the  data  structures  in  an 
"intelligent"  way.  We  discuss  here  three  criteria  for  evaluating  a  knowledge 
representation: 

1.  Can  it  be  matched  with  the  actual  behavior  of  subjects?  There  are 
two  sides  here:  (a)  Can  the  representation  be  derived  from  behavior,  and  (b) 
Can  i t  be  used  to  predict  behavior? 

2.  When  the  representation  is  used,  does  it  contain  the  necessary  and 
sufficient  information  for  successfully  performing  the  tasks  it  is  designed  to 
handle? 

3.  Is  it  efficient,  i.e.,  sufficiently  fast  to  be  used  on-line? 

We  briefly  comment  on  the  three  criteria: 


1.  Matching  the  Representation  and  Behavior 


(a)  If  the  representation  can  be  derived  from  behavior,  it  can  be  useful 
in  designing  individualized  instructions,  i.e.,  instructions  that  fit  the  way 
a  particular  person  is  conceptualizing  a  task.  (We  know  from  our  own 
research,  e.g.,  Baggett,  1983;  Baggett  &  Ehrenfeucht,  1985,  that  such  a 
conceptual  match  is  important  for  performance.  Lack  of  this  ability  in 
computerized  instruction  thus  far  is  one  shortcoming  given  by  Sleeman  and 
Brown,  1982.  Its  presence  can  be  considered  evidence  that  the  program  is 
indeed  "intelligent".) 

(b)  The  ability  to  use  the  representation  to  predict  behavior  gives  the 
representation  psychological  validity. 

We  note  that  in  a  production  system  representation  (e.g.,  Anderson, 
Kline,  &  Beasley,  1979;  Davis,  Buchanan,  &  Shortliffe,  1976;  Davis  &  King, 
1977;  Hedrick,  1976;  Lenat,  1982;  Rychener,  1976;  Shortliffe,  1976;  Vere, 
1977;  Waterman,  1970),  (a)  is  difficult  and  (b)  is  easy.  In  a  semantic 
network  representation  (e.g.,  Anderson  &  Bower,  1973;  Brachman,  1979; 
Carbonell,  1970;  Carbonell  &  Collins,  1974;  Hendrix,  1976;  Norman  &  Rumelhart, 
1975;  Quillian,  1978;  Stefik,  1980;  Walker,  1976;  Woods,  1975;  Woods  et  al., 
1976),  (a)  is  easy  and  (b)  is  difficult. 

2.  Relativity  of  the  Knowledge  Representation 

In  evaluating  what  is  a  good  knowledge  representation,  we  keep  in  mind 
that  there  is  no  absolute  or  best  or  ideal  one.  (This  fact  was  also  discussed 
by  Wilensky,  1984).  Evaluation  of  the  knowledge  representation  must  be  done 
relative  to  the  task  (i.e.,  the  context  or  environment).  There  are  two  points 
here:  "~ 


(a)  The  knowledge  representation  should  not  have  extraneous  elements, 

i.e.,  elements  never  used  in  a  given  group  of  tasks. 

(b)  The  knowledge  representation  should  be  complete,  i.e.,  within  the 

group  of  tasks,  all  actions  that  can  make  a  difference  in  the  outcome  must  be 
accounted  for  (included). 

3.  Efficiency  of  the  Representation 

For  a  practically  designed  tutoring  system,  efficiency  is  an  important 
issue.  It  can  be  divided  into  two  components: 

(a)  Non-laboratory  tasks,  for  which  it  is  worthwhile  to  have  a  tutoring 

system,  are  fairly  complex.  So  the  size  of  the  data  base  used  for  the 

knowledge  representation  can  be  a  crucial  element  that  determines  the  success 
or  failure  of  the  system. 

(b)  An  interactive  tutoring  system  requires  real-time  processing  of 
data.  Subjects  should  not  have  to  wait  long  for  its  response  (Anderson,  1984; 
Shneiderman,  1980).  So  the  algorithms  for  data  processing  have  to  be  fast. 

In  III.  we  propose  a  knowledge  representation  which  seems  to  satisfy  the 
three  criteria  above. 
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III.  Representing  Knowledge  in  the  Theoretical  Framework 
A.  The  Representation 

The  13  diagrams  on  pages  A1  through  A5  (Appendix  1)  represent  a 
fictitious  (not  experimentally  derived)  adult's  conceptualization  of  a 
flashlight,  as  viewed  in  our  framework.  The  seven  diagrams  on  pages  Cl 
through  C3  (Appendix  2)  represent  a  fictitious  child's  conceptualization,  also 
viewed  in  our  framework.  Here  is  an  explanation  of  the  conventions  used  in 
the  diagrams: 


A  - ^  B  means  B  is  a  subconcept  of  A. 

A - B  n.ians  A  has  a  pointer  to  B. 


Concepts  in  circles  are  abstract. 

Concepts  in  triangles  are  motoric  (actions).  They  are  denoted  by  capital 
letters. 

Concepts  in  squares  are  visual.  They  are  executable  by  the  visual  processor. 
We  label  them  V^.  The  values  for  each  are  given  in  Figure  1. 

Concepts  in  ovals  are  linguistic.  They  contain  verbal  labels.  In  the 
diagrams,  some  subconcepts  do  not  have  linguistic  components.  This 

corresponds  to  the  fact  that  many  people  do  not  have  ready-made  names  for 
many  parts. 

Verbal  explanations  to  the  right  of  each  diagram  are  not  part  of  the  concepts. 
They  are  merely  explanations  for  the  reader. 

The  division  of  the  adult's  conceptualization  into  13  diagrams  and  the 
child's  into  7  is  done  solely  for  the  purpose  of  readability.  Each 

conceptualization  could  just  as  well  be  one  diagram. 

Now  we  shall  discuss  some  properties  of  the  diagrams  in  detail. 

In  diagram  2.  structure  (adult),  V~  represents  the  flashlight  as  the 
person  sees  it.  (See  Figure  1.)  A-1  has  pointers  to  both  V.  and  V5-  The 

pointers  indicate  that,  if  the  person  unscrews  the  cap,  he  or  ?he  expects  to 

see  a  case  with  batteries  (V.)  and  a  front  part  with  protector  and  bulb  ( V,- ) • 

V.  and  Vr  are  concepts  executable  by  the  visual  system,  so  that  the 
expectation  can  be  verified.  For  example,  if  the  actual  flashlight  whose  cap 
was  unscrewed  did  not  contain  batteries,  execution  of  V.  would  fail  to  detect 
the  batteries. 

The  pointer  from  A  to  V2  indicates  that  if  the  person  screws  the  cap 

onto  the  case  with  batteries  V.,  the  person  expects  to  see  a  complete 

flashl ight. 

This  pattern  repeats  in  many  diagrams.  For  example,  for  the  adult,  each 
action,  besides  having  a  verbal  subconcept,  also  has  one  or  more  visual 
subconcept( s)  which  can  be  treated  as  arguments  for  the  action.  The  action 
also  has  a  pointer  to  one  or  more  visually  recognizable  concepts,  which 
represent  the  objects  seen  as  the  result  of  the  action. 


Other  pointers  from  an  action  point  to  abstract  concepts.  An  example  is 
in  diagram  2.  structure  (adult).  Here,  a  pointer  goes  from  action  A  to  C3,  an 
electrical  connection.  This  means  that  the  action  of  screwing  on  the  capJ 
should  cause  the  electrical  connection  between  bulb  and  battery. 

In  diagram  3.  functionality  (adult),  we  see  a  pointer  going  from  abstract 
concept  30  to  L.  Thi s  means  tEe  person  expects  the  light  to  be  off  because 
the  electric  circuit  is  open.  A  pointer  from  abstract  concept  29  to  L 
indicates  the  light  is  one  because  the  circuit  is  closed. 

These  examples  show  that  the  actual  meaning  of  pointers  (causal, 
expectations,  etc.)  depends  on  the  types  of  concepts  they  are  linking.  In 
general,  a  meaning  for  the  pointer  will  depend  on  the  context  in  which  it  is 
used.  In  an  example  later  in  this  paper,  some  pointers  will  indicate  a 
temporal  sequence  of  actions.  Even  in  diagram  2.  structure,  pointers  between 
A_i  and  V.  and  V5  could  have  a  temporal  interpretation:  "First  unscrew  the 
cap,  and  then  look  at  the  case  with  batteries  and  the  front  part." 

Similar  ambiguities  occur  also  in  interpreting  other  parts  of  the 
diagrams.  Concept  A  can  be  interpreted  as  a  relation,  i.e.,  "the  front  part 
is  screwed  on  the  case  with  batteries,"  as  well  as  the  action  of  screwing  the 
front  part  on  the  case.  Similarly,  in  4.  case  with  batteries,  B  can  be 
interpreted  as,  "batteries  are  in  the  case,"  as  well  as  the  action  of  putting 
the  batteries  in  the  case. 

This  distinction  of  relation  versus  action  that  causes  the  relation  is 
not  represented  in  the  concept.  The  distinction  occurs  only  when  the  concept 
is  used.  For  example,  in  assembling  or  disassembling  a  flashlight,  the 
concepts  would  actually  be  executed,  and  then  clearly  they  are  actions.  If 
the  concept  is  used  as  a  blueprint  for  a  verbal  description  of  the  structure 

of  a  flashlight,  the  concepts  A,  A"1,  etc.,  are  verbally  represented  as 

relations  between  parts  of  the  object. 

Turning  now  to  a  general  overview  of  the  adult  and  child  conceptualiza¬ 
tions,  we  can  summarize  their  knowledge  as  follows.  The  adult  has  a  fairly 
detailed  knowledge  of  the  structure,  a  reasonable  knowledge  cf  the  circuitry, 
and  some  understanding  of  the  relationship  between  the  structure  and  the 
circuitry.  The  child  has  a  much  more  limited  knowledge  of  the  structure  and 
no  knowledge  about  the  circuitry.  The  child  also  has  a  restricted  verbal 
terminology.  We  can  also  notice  (in  6.  case  (child))  tl.at  the  child  does  not 

view  the  contact  bar  as  a  part  of  the  switch,  but  simply  as  a  part  of  the 

body . 


Another  point  needs  to  be  mentioned  here.  Some  things  are  left  out  of 
the  conceptualizations  presented.  For  example,  there  is  no  mention  of  the 
type  of  batteries  or  of  electric  current.  Such  omissions  do  not  mean  that  the 
person  does  not  have  a  knowledge  of  different  types  of  batteries,  etc.  It 
only  means  that  this  knowledge  is  not  incorporated  as  a  subconcept  of  the 
concept  of  flashlight.  (See  also  III.E.7  on  world  knowledge.) 


B.  Use  of  Concepts 

1.  Actions  and  Strategies 


The  concept  of  the  flashlight  as  presented  in  the  13  diagrams  allows  our 
fictitious  adult  to  perfonj  many  tasks,  including  repair  tasks.  For  example, 
the  sequence  of  actions  A  B  1  removes  the  batteries  from  the  flashlight,  and 
the  sequence  BA  puts  them  back.  This  allows  the  person  to  change  the 
batteries.  Similarly,  the  person  can  change  the  bulb,  or  replace  some  other 
removable  part  of  the  flashlight. 

The  conceptualization  presented  above  also  allows  the  person  to  make  some 
diagnoses  of  a  malfunction.  For  example,  if  the  light  doesn't  go  on  after  the 
switch  is  pushed  forward,  the  person  knows  something  is  wrong,  because  the 
expected  result,  as  shown  by  a  pointer  from  G  to  L  (in  3.  functionality)  is 
that  the  light  should  be  on.  A  pointer  from  29  to  L  (in  the  same  diagram) 
gives  the  information  that  the  closed  circuit  causes  the  light  to  be  on. 
Closer  examination  of  a  circuit  (in  the  last  adult  diagram,  p.  A5)  shows  what 
connections  have  to  be  made,  and  which  elements  are  part  of  the  connections. 

In  addition,  for  example,  the  pointer  in  5.  front  part  with  protector  and 
bulb  from  C  to  Cp  indicates  that  snapping  the  protector  into  the  back  of  the 
refl ector  should*1  form  , connection  Cp.  This  information  could  lead  to  the 
following  actions:  A  XC  CA.  These  actions  would  remove  the  front  part, 
remove  the  protector  with  bulb,  snap  it  in  again,  and  screw  on  the  front  part 
again.  Such  a  sequence  of  actions  could  possibly  correct  a  poor  contact 
between  the  bulb  and  the  collar. 

For  comparison,  our  fictitious  child  has  far  more  limited  possibilities 
for  actions.  When  there  is  a  malfunction,  the  child  is  basically  limited  to 
pushing  the  switch  on  and  off,  and  replacing  the  batteries. 

This  brings  up  again  the  question  of  background  knowledge:  Can  a  person 
perform  some  actions  based  on  general  background  knowledge?  We  think  (we  do 
not  have  experimental  evidence!  tHat  background  knowledge  is  used  in  the 
following  way:  Some  specific  concepts  from  background  knowledge  can  be 

incorporated  into  the  concepts  that  are  processed.  This  means  the  person 
changes  his  conceptualization  of  the  flashlight.  Then  the  person  bases  his 
actions  on  his  modified  conceptualization. 

But  in  order  to  perform  even  very  simple  specific  tasks,  the  person  needs 
a  strategy.  Without  one,  the  actions  available  would  not  be  performed,  or 
would  be  performed  in  a  random,  aimless  order.  We  think  that  application  of  a 
strategy  consists  of  two  parts: 

1. )  Forming  a  plan  of  action,  which  basically  consists  of  performing  some 
mental  operations;  and 

2. )  Carrying  out  the  plan. 

Of  course  the  two  parts  do  not  have  to  be  done  in  the  order  given:  A 
partial  plan  can  be  formed,  and  partial  execution  can  be  done. 

In  our  framework,  strategies  are  concepts,  with  an  overall  structure 
similar  to  the  structure  of  an  object  such  as  the  flashlight  given  above.  The 
difference  is  that  they  will  contain,  among  other  elements,  executable  mental 
actions.  Executable  mental  actions  are  treated  in  the  same  way  as  executable 
motoric  actions,  with  one  difference.  Motoric  actions,  when  executed,  operate 
on  physical  objects.  Mental  ations,  when  executed,  operate  on  concepts. 
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In  the  examples  below,^we  shall  indicate  executable  mental  actions  by 
putting  them  in  octagons:  (m)  .  We  shall  start  with  an  example  of  a  strategy 
for  removing  the  bulb  from  tKe  flashlight  (Figure  2),  and  later  generalize  it 
to  removing  a  part  (subassembly)  from  an  object. 


M. .  concept  of  removing 

bulb  from  flashlight 

•  Form  a  plan. 

•  Carry  out  the  plan. 

.  Forget  the  plan. 

P.  Abstract  plan 

(there  is  nothing  in  it) 


Figure  2 


Mental  operations  can  be  defined  in  terms  of  subconcepts,  the  subconcept 
relation,  and  pointers.  Below,  we  give  the  definitions  of  M2,  M^,  and  M^. 


Definitions. 


XPrel  iminary^definition)  A  chain  between  (uj  and  |  w  |  is  a  sequence 

A\  rnrrnnir  r  , h  +■  h  =>  +  * '  l— J 


of  concepts  such  that 

A 


and 


and 


each  /i(^  points  to  a  subconcept  of  \  ,  i.e.. 


KiN - 


M  K- 


Here  i^s  an  example  taken  from  our  adult's  conceptualization. 

and 


■e  is  an  example  taken  from  our 

.  I*: 


_  is  a  chain  between 

reason  it  is  a  chain  is  shown  in  Figure  3. 


Vij-  The 


Figure  3 


page  9 


Definition  of  M^.  consists  of  2  steps. 

Step  1.  Find  a  chain  from  V,  to  V. .. 

Step  2.  Make  all  elements  of  the  cnain  subconcepts  of  P. 

(On  paper,  it  is  shown  as  follows:  Solid  arrows  are  drawn  from  concept  P  to 

elements  of  the  chain.  See  Figure  4.) 


Figure  4 

Definition  of  M  .  Execute  all  subconcepts  and  subsub.... concepts  of  P  in 
order,  constrained  as  follows: 

(a)  A  subconcept  must  be  executed  before  its  concept;  and 

(b)  If  one  concept  points  to  another,  the  second  one  must  be  executed 
after  the  first. 

Definition  of  M. .  Remove  all  subconcepts  of  P.  (On  paper,  it  is  shown  as 
fol 1 ows :  Prase  all  solid  arrows  that  start  in  P.) 

The  concept  M.  is  a  strategy  specific  for  the  flashlight.  This  can  be 
seen  by  observing  that  subconcepts  V-  and  V , .  are  common  subsubconcepts  for  M, 
and  for  1  (on  p.  Al),  the  adult's  concept  of at1 ashl ight. 

(1)  execution  of  (mental,  which  would  lead  to  modification  of  P  (making  P 
concrete).  This  gives  the  concept  shown  in  Figure  4. 


(2)  execution  of  M,,  which  consists  of  the  following  sequence  of  physical 
actions: 


1. 


2. 


Look  at  (locate)  flashlight. 


Unscrew  cap. 
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Si 


Look  at  front  part. 

Tilt  and  remove  protector  from  back  of  reflector. 
Look  at  protector  with  bulb. 

Take  bulb  out  of  protector. 


Look 


(3)  execution 
in  Figure 


at  bulb. 

of  M.  (mental,  which  restores  the  original  abstract  P  (shown 
2  above.)) 


We  observe  three  things: 

(1)  The  execution  of  makes  the  plan  P  concrete  (in  terms  of  Baggett  & 
Ehrenfeucht,  1982). 

(2)  In  the  execution  of  M3  (carrying  out  the  plan),  motoric  components  direct 
the  actions,  and  visual  components  direct  the  attention  of  the  person 
performing  the  action. 

13)  Execution  of  M.  is  an  example  of  an  abstraction.  It  removes  all 
subconcepts  from  p,  returning  it  to  its  original  abstract  status. 

We  shall  now  show  how  strategies  for  removing  other  parts  of  the 
flashlight  can  be  obtained  from  the  strategy  shown  above,  by  transfer  of 
learning. 


In  order  to  remove  the  batteries  of  the  flashlight,  one  needs  only  to 
replace  V..  by  V7  in  Figure  2.  The  reader  may  check  that  the  execution  of  such 
a  modified ^concept  would  indeed  remove  the  batteries.  Similarly,  replacing  VJ4 
by  V2Q  would  lead  to  removing  the  transparent  disc. 

We  note  that  replacement  of  V.4  by  V22  would  lead  to  a  concept  that  would 
fail  to  execute,  because  there  is  no  chainTetween  V2  and  V22  in  the  concept  of 
the  flashlight.  (The  collar  is  nondetachable.)  But  of  course  the  metal  part, 
which  the  collar  is  a  part  of,  i_s^  detachable  and  can  be  removed  from  the 
flashlight  by  replacing  V  14  by  Vlg. 

We  note  also  the  role  of  We  assume  for  a  moment  that  M4  is  not 

present.  (This  would  be  the  situation  depicted  in  Figure  2,  but  with  no  M4. ) 
This  strategy  would  lead  to  the  correct  removal  of  the  bulb  from  the 

flashlight.  But  attempting  the  first  editing  operation,  namely,  replacing  V,4 

by  V7,  would  give  a  different  result  than  before.  Namely,  execution  of  Pl2 
would  simply  add  new  subconcepts  to  P,  which  already  ha_s  subconcepts  assigned, 
from  previous  use.  It  would  lead  to  a  sequence  of  actions  which  would  remove 
the  batteries  and^  remove  the  bulb.  This  Is  an  example  of  carrying  unnecessary 
actions  from  a  previous  task  to  a  new  task  during  transfer  of  learning.  The 

error  is  rather  typical,  and  in  this  case  it  is  solely  due  to  not  performing 


si 


the  abstraction  operation  (th'.t  is,  to  not  removing  elements  from  the  plan  P 
from  the  previous  task). 


A  general  strategy  for  removing  a  part  of 
take  again  our  original  Figure  2  and  make  V~ 
Figure  5.  L 


an  object  is  as  follows.  We 
and  V14  undefined,  as  shown  in 


Figure  5 


M.  .  concept  of  removing 
1  a  part  from  an  object 
M2*  Form  a  plan. 

Mj.  Carry  out  the  plan. 

M^.  Forget  the  plan 
P.  Abstract  plan 

u  and  w  are  undefined 


That  u  and  w  are  undefined  means  that  the  type  of  u  and  w  is  still  a  visually 
executable  concept,  but  they  are  not  executable.  That  is,  they  do  not  have  a 
value  that  actually  corresponds  to  a  visual  image. 

Obtaining  Figure  5  from  Figure  2  is  an  example  of  generalization,  as  given 
in  Baggett  &  Ehrenfeucht,  1982. 

We  observe  that  the  concept  in  Figure  5  is  not  flashlight-specific.  (It 
does  not  have  any  subconcepts  in  common  with  the  flashlight  concept.)  It  can 
be  appTTed  to  the  conceptualization  of  any  object. 

This  concept  of  removing  a  part  is  still  unsatisfactory  in  some  respects. 
In  particular,  the  strategy  it  represents  does  not  tell  what  to  do  if  execution 
fails,  for  example,  if  a  plan  is  not  found,  or  if  execution  of  the  plan  fails, 
for  one  reason  or  another. 


For  an  executable  concept  X  we  will  denote  by  X'  the  failure  of  execution. 
(This  convention  can  be  seen  in  3.  functionality  (adult),  where  L'  denotes 
failure  to  detect  the  light.)  When  there  are  pointers  going  out  from  an 
executable  concept,  we  will  make  a  distinction  about  which  pointers  have  to  be 
followed  when  the  execution  is  a  success  versus  when  the  execution  is  a 
failure.  We  graphically  represent  this  as  follows: 


— - o 
o 


next  concept,  if  X  is  successful 
next  concept,  if  X  is  unsuccessful 


(We  are  not  introducing  an  new  type  of  pointer.  We  are  simply  Indicating  which 
pointer  has  to  be  followed,  depending  on  the  success  or  failure  in  the 
execution  of  concept  X.)  We  note  that  this  is  the  simplest  (and  most 
Important)  example  of  automatic  decision  making,  as  viewed  in  our  framework. 


In  the  example  in  Figure  6,  we  have  added  new  linguistic  subconcepts  to 
our  concept  of  removing  parts.  (Linguistic  concepts  are  executable  in  our 
framework.)  So  now,  application  of  the  strategy  in  Figure  6  to  any  object 
would  lead  to  one  of  three  outcomes: 


IT  CANNOT 


I  CANNOT  DO  IT 


Figure  6 


(1)  removing  a  part,  or 

(2)  verbal  output,  "it  cannot  be  done,"  for  the  case  where  a  plan  cannot  be 
formed;  or 

(3)  verbal  output,  "I  cannot  do  it,"  for  the  case  where  the  plan  was 
successfully  formed,  but  execution  of  the  plan  failed,  that  is,  one  of  the 
motoric  or  visual  components  of  the  concrete  concept  P  failed  to  execute. 

2.  Inferences 


Our  framework  does  not  include  any  system  of  formal  rules  of  logic.  There 
are  two  reasons  for  this.  First,  we  doubt  the  psychological  validity  of 
logical  rules,  as  formulated  in  logical  systems  (people  don't  seem  to  think 
according  to  the  rules  of  logic).  Second,  we  don't  see  the  need  for  such  a 
set  of  rules  in  our  framework. 

Basic  inferences  are  directly  represented  by  pointers  in  our  knowledge 
representation.  There  is  one  difference  between  our  approach  and  logical 
inferences:  In  our  representation,  a  pointer  from  G  (push  switch  forward)  to  L 
(light  is  on),  shown  in  3.  functionality  (adult),  does  not  indicate  that 
whenever  the  person  pushes  the  swi tch  Forward,  the  light  will  be  one.  It 
indicates  only  that  after  pushing  the  switch  forward,  the  person  should  expect 
the  light  to  be  on,  and  should  try  to  confirm  the  expectation  in  this  case  by 
looking  for  the  light. 

Forming  a  chain  of  reasoning  is  not  obtained  by  application  of  any  rules 
of  logic,  but  by  the  search  for  a  chain  of  concepts,  as  in  the  example  above. 
The  definition  of  chain  given  above  is  just  an  example.  Different  definitions 
of  chains  would  correspond  to  different  methods  of  reasoning. 

Another  example  of  inference  is  given  in  Figure  6,  when  the  person  says, 
"It  cannot  be  done."  In  this  case,  the  inference  is  very  simple.  "It  cannot 
be  done"  expresses  the  fact  that  the  plan  for  action  cannot  be  formed,  and 
therefore  action  is  impossible.  This  shows,  among  other  things,  that  all 
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Inferences  are  not  universal  (in  our  framework).  Rather,  they  are  situation- 
dependent.  The  conclusion,  "It  cannot  be  done,"  would  be  reached  by  our 
fictitious  child,  when  the  child  is  asked  to  remove  the  bulb  from  the 
flashlight.  But  it  would  not  be  reached  by  our  adult. 

C.  Instructions 

Let  us  assume  that  the  adult  above  wants  to  teach  the  child  how  to  fix  a 
malfunctioning  flashlight,  or  to  do  some  other  task  related  to  the  flashlight. 
(For  convenience  we  henceforth  refer  to  the  adult  as  A  and  the  child  as  B.)  We 
consider  that  there  are  two  essential  elements  in  this  process: 

(1)  A  needs  to  find  out  what  B  knows.  This  means  A  should  construct  a  concept 
that  is  as  close  as  possible  to  the  concept  B  has.  Ideally,  A  will  have, 
besides  his  own  concept  of  the  flashlight,  also  the  concept  of  the  flashlight 
that  B  has. 

(2)  A  has  to  instruct  B.  The  method  of  instruction  within  our  framework  will 
be  just  a  strategy,  requiring  planning  and  actions.  A  can  use  different 
strategies,  depending  on  the  situation,  but  all  will  have  one  thing  in  common: 
The  objective  of  the  instruction  is  to  make  B  modify  his  concept  in  some 
specific  way.  (In  our  framework,  this  means  that  B  has  learned.) 

There  are  many  possibilities  for  A  to  achieve  (1)  mentioned  above.  One  is 
to  give  B  some  task  to  do  involving  the  flashlight  and  see  how  B  performs. 

In  our  example,  suppose  A  gives  B  a  flashlight  with  a  burned  out  bulb  and 
says,  "Can  you  fix  it  for  me?"  Here  are  some  hypothetical  reactions  of  B, 
based  on  B's  conceptualization:  B  pushes  the  switch  forward  and  backward  a  few 
times  with  no  result.  He  then  says,  "The  batteries  are  bad.  Do  you  have  any 
good  batteries?"  After  being  given  new  batteries,  B  replaces  the  old  batteries 
with  the  new  ones  and  pushes  the  switch  several  more  times,  again  with  no 
results.  He  says,  "I  cannot  fix  it.  It  is  broken." 

We  notice  this  simple  interaction  gives  a  pretty  fair  summary  of  B's 
conceptualization,  both  in  terms  of  the  actions  that  B  can  perform,  and  his 
knowledge  about  how  the  flashlight  works. 

But  A's  interpretation  of  B's  behavior  depends  on  A's  conceptualization  of 
a  flashlight.  In  order  to  obtain  B's  conceptualization,  A  needs:  (A)  his  own 
conceptualization  and  (2)  an  observation  of  B's  behavior.  This  brings  up  a  new 
mechanism  not  mentioned  in  the  above  example.  The  mechanism  is  building  new 
concepts  (in  this  case,  the  conceptualization  of  B),  based  on  the  concepts  that 
A  already  has  and  on  input  stimuli,  an  observation  of  the  behavior  of  B. 
Treatment  of  tTvTs  problem  will  be  discussed  in  part  III.D.4  of  this  addendum. 

Regarding  (2)  above,  the  method  of  teaching  is  treated  as  any  other 
strategy:  it  involves  planning  and  execution  of  the  plan. 

We  discuss  here  a  small  but  important  technical  point:  How,  in  our 
representation  of  knowledge,  will  actions  performed  by  different  people  be 
represented? 

Suppose  John  changed  the  batteries.  This  information  would  be  represented 
as  in  Figure  7. 


••  '  * 

>  v 
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1.  John  changed  batteries,  or 
Batteries  were  changed  by 
John. 

2.  the  person  John 

3.  batteries 

X.  action  of  changing 


Figure  7 


This  representation  differs  from  the  representation  used  in  semantic 
networks,  where  John  and  the  action  would  be  directly  connected  by  a  labelled 
pointer,  indicating  that  John  is  the  actor  of  the  action,  it  also  differs  from 
the  propositional  approach,  in  which  John  would  be  treated  as  an  argument  of 
the  action  in  the  proposition. 

Our  approach,  on  the  other  hand,  is  the  one  used  in  standard  linguistic 
processing,  where  simple  sentences  are  divided  into  noun  phrase  and  verb 
phrase: 


NP  corresponds  to  John,  and  VP  corresponds  to  an  action,  together  with  its 
arguments. 

Thus,  our  person  A  would  represent  the  fact  that  B  replaced  the  batteries 
as  in  Figure  7  (with  B  substituted  for  John). 

D.  An  Overview  of  the  Proposed  Programs 

The  (planned)  program  will  consist  of: 

(1)  a  data  base 

(2)  special  purpose  procedures 

(a)  concept  processing  procedures 

(b)  output  procedures 

(3)  an  English  parser 

(4)  a  concept  building  procedure 

(5)  a  driver 

Each  of  these  will  be  briefly  discussed. 
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(1)  The  data  base 

The  data  base  will  contain  knowledge  represented  as  concepts,  as  shown 
above.  It  will  contain  a  concept  of  the  object,  with  the  amount  of  detail 
corresponding  to  the  amount  of  detail  used  in  the  description  of  person  A 
above.  It  will  also  contain  repair  concepts  that  would  correspond  to  the 
concept  of  removing  a  part  of  the  object,  as  shown  above. 

We  have  estimated  that  a  detailed  representation  of  a  concept  for 
repairing  the  flashlight,  one  that  would  cover  a  malfunction  in  any  part  of  the 
electric  circuit  or  in  a  mechanical  part,  or  in  a  combination,  is  approximately 
of  the  same  size  as  the  description  of  the  flashlight  given  for  the  adult.  (We 
use  here  the  number  of  subconcepts  as  a  measure  of  the  size.) 

The  data  base  will  also  contain  a  teaching  strategy.  Its  size  will  depend 
on  the  strategy  used.  Also,  other  concepts  will  be  built  by  the  program  and 
added  to  the  data  base  during  interaction  of  the  program  with  the  subject. 

The  concepts  which  will  be  put  in  the  data  base  (conceptualizations  of  the 
objects,  repair  strategies,  etc.)  will  be  experimentally  derived  and  tested 
(and  not  fictitious,  namely,  invented  by  the  experimenter,  as  were  the  concept 
of  the  flashlight  given  above). 

(2)  Special  purpose  procedures 

(a)  concept  processing  procedures 

The  program  will  contain  basic  procedures  corresponding  to  basic  mental 
operations  such  as  M„,  M3,  and  M4  above.  (A  special  procedure  for  M,  is  not 
needed,  because  execution  of  M,*  consists  of  execution  of  M~,  M3,  and  ) 
Among  the  basic  procedures,  there  will  be  ones  for  concept  comparison,  which 
have  not  been  illustrated  in  the  examples  above.  (All  special  purpose 
procedures  will  be  data-driven.  That  is,  the  order  in  which  they  are  evoked 
will  be  determined  by  concepts  in  the  data  base.) 

(b)  output  procedures 

The  program  will  also  contain  procedures  that  control  visual  and  verbal 
displays  on  the  screen.  These  will  also  be  data-driven,  in  the  following  way: 
Each  motorically  executable  concept  (as  in  the  examples  above)  will  correspond 
to  having  one  the  screen  a  segment  of  videotape  showing  performance  of  the 
action.  Visual  concepts  will  correspond  to  still  pictures.  Linguistic 
concepts  will  provide  the  text.  Thus,  the  action  of  the  program  in  executing 
concept  M1  (in  figure  4),  stored  in  the  data  base,  will  be  as  follows: 

(1)  Execution  of  M„:  Search  through  data  base,  constructing  a  chain,  and 
connect  the  chain  to  P. 

(2)  Execution  of  M3:  Show  on  the  screen  the  sequence 

(i)  still  picture  of  flashlight 

(ii)  moving  video  of  unscrewing  cap 

(iii)  still  picture  of  front  part 


(1v)  moving  video  of  tilting  and  removing  protector  from  back  of 
reflector 

(v)  still  picture  of  protector  with  bulb 

(vi)  moving  video  of  taking  bulb  out  of  protector 

( vi i )  still  picture  of  bulb 

We  note  that  this  part  of  the  program  is  the  most  task-specific.  Clearly, 
the  visual  display  will  depend  on  the  object  to  be  repaired. 

(3)  English  parser 

The  English  parser  will  take  the  subject's  input  and  construct  a  very 
simple  knowledge  representation  of  it  (in  the  form  of  a  concept),  as  is  shown, 
for  example,  in  Figure  7. 

At  the  present  time  some  students  of  A.  Ehrenfeucht  in  the  Computer 
Science  Department  are  testing  a  rather  sophisticated  English  parser,  written 
in  LISP,  that  possibly  can  be  used  for  this  purpose. 

(4)  Concept  building  procedure 

This  procedure  is  the  most  complex  part  of  the  whole  program.  It  will 

take  as  its  input  Lne  sequence  of  interactions  between  a  subject  and  the 

program,  and  the  concept  of  the  object  from  the  data  base,  and  return  to  the 

data  base  a  new  concept,  "how,  at  a  particular  point  in  time,  the  subject 

conceptualizes  the  object,"  Similarly,  when  taking  from  the  data  base  the 
concept  of  "how  to  repair  the  object",  it  would  return  the  concept,  "how  the 
subject  thinks  the  object  should  be  repaired." 

To  be  a  little  more  specific,  let  us  assume  (still  using  the  flashlight 
example)  that  the  following  sequence  has  been  shown  on  the  screen: 

still  picture  of  flashlight 
moving  video  of  unscrewing  cap 
moving  video  of  removing  batteries 
moving  video  of  putting  batteries  back  in 
moving  video  of  screwing  cap  back  on 

The  subject's  response,  typed  on  the  terminal,  is,  "I  cannot  unscrew  it." 
From  the  poin|  o£  view  of  internal  processing,  the  special  purpose  output 
procedures  A"1  B  1  BA  were  evoked.  The  concept,  call  it  101,  shown  in  Figure 
8,  would  be  created  by  the  parser. 

101.  Subject  cannot  unscrew 
the  cap. 

(A-^) '  action  of  unscrewing  the 
cap  was  unsuccessful 


Figure  8 
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The  sequence  A"  ,  B  ,  B,  A,  101  is  the  actual  input  for  the  concept 
building  procedure.  After  this  interaction,  the  concept  of  how  the  subject 
conceptualizes  the  flashlight  would  have  A-1  as  a  nonexecutabl e  subconcept. 

The  algorithm  used  in  the  concept  building  program  will  be  a  modification 
of  the  cluster  analysis  algorithm  described  in  Perry  (1983). 

(5)  A  driver 

A  driver  will  synchronize  the  action  of  other  procedures  and  keep  track  of 
concepts  in  the  data  base  which  are  processed  at  a  given  time. 

E.  Comparison  of  the  Proposed  Knowledge  Representation  with  Others 

The  knowledge  representation  we  proposed  seems  to  satisfy  the  criteria  in 
II. B.,  1,  2,  and  3,  above.  Further,  each  of  the  three  elements  has  been 
partially  tested  in  practice.  Some  of  the  details  (e.g.,  for  matching  behavior 
in  a  repair  task  with  the  representation)  were  presented  in  Baggett  (1984).  We 
give  here  a  brief  summary. 

l.a.  We  can  construct  a  subject's  hypothetical  conceptualization  from  the 
subject's  performance  (Baggett,  1983,  1984;  Baggett  &  Ehrenfeucht,  1985. 

Baggett  &  Perrig,  1985). 

1. b.  We  have  been  able  to  manipulate  subjects'  assembly  performance  by 
changing  the  representation  of  an  object  presented  in  an  instructional 
videotape.  (These  were  the  “typical''  vs.  "minority"  conceptualization  results 
presented  in  Baggett  &  Ehrenfeucht,  1985).  This  means  that  our  representa¬ 
tional  approach  is  psychologically  valid  and  predictive. 

2.  Regarding  relativity  of  the  representation,  whe?'  we  match  performance 
of  a  subject  to  the  representation,  we  know  what  is  never  used  and  what  is 
needed  to  make  a  complete  representation.  Further,  we  know  how  to  test  for 
extraneous  material  and  completeness  (Baggett,  1984). 

3.  Concerning  the  efficiency  of  the  representation,  we  know  about  the 
size  of  the  data  base  from  objects  containing  nearly  100  pieces  and  over  100 
physical  connections.  We  also  know  about  the  algorithms  which  will  operate  on 
the  data  base. 

First,  in  our  assembly  tasks  we  have  worked  with  conceptual izations 
(representations)  from  hundreds  of  subjects.  The  data  bases  constructed  were 
fairly  small.  Preliminary  results  from  our  repair  tasks  show  that  the  data 
bases  will  be  of  similar  size.  Second,  the  algorithms  we  have  used  (Perry, 
1983)  have  been  fast  enough  for  the  tasks  we  have  analyzed  thus  far.  They  have 
not  been  tested,  however,  in  real-time  use,  that  is,  for  processing  that  must 
be  done  while  the  subject  works  at  the  terminal. 

4.  A  Comparison  of  Our  Representation  with  KRL 

Henry  Halff  (personal  communication,  1983)  asked  whether  KRL,  a  knowledge 
representation  language  (Bobrow  &  Winograd,  1977),  has  similarities  with  our 
representation.  We  very  briefly  discuss  the  similarities  and  also  the 
differences. 


a.  In  KRL  knowledge  is  organized  around  conceptual  entities.  This  is 
similar  in  our  approach  in  which  conceptual  entities  are  primitives.  However, 
KRL  stresses  descriptions  associated  with  the  entities,  which  we  do  not  have. 

b.  Partial  knowledge  is  represented  in  both  approaches.  We  have 
different  decompositions  (conceptualizations);  KRL  has  different  descriptors. 
KRL  represents  different  viewpoints;  we  have  different  representations  for 
different  purposes. 

c.  Both  approaches  are  object-oriented.  But  our  approach  treats  reasoning 
as  an  obj ect,  the  same  as  other  concepts.  For  us,  a  strategy,  or  the  logic 
used,  etc.,  are  objects  just  as  any  other  concepts  are. 

d.  KRL  clusters  its  information  according  to  its  use  by  processors.  We 
do  not.  For  us,  a  concept  contains  a  mixture  of  elements,  to  be  used  by 
various  processors. 

e.  KRL  is  a  language.  We  are  not  trying  to  develop  a  language  for  our 
knowledge  representation.  We  can  use  any  number  of  existing  languages. 

The  most  important  difference  between  KRL  and  our  approach  is  that  KRL 
works  with  text  objects  and  descriptions,  while  in  our  data  base  visual 
elements  are  an  essential  part.  Some  objects  in  our  data  base  do  not  even  have 
a  verbal  description,  except  for  a  simple  label  for  identification.  For 
example,  in  our  sequencing  of  the  shots  of  a  videotape  (Baggett,  1984),  there 
are  no  linguistic  descriptions  of  what  the  sequences  show,  and  such 
descriptions  are  also  not  needed. 

5.  Our  Approach  and  Frames  and  Semantic  Networks 

Conceptually,  our  approach  is  closer  to  frames  (Minsky,  1975;  Bobrow  & 
Winograd,  1977;  Goldstein  &  Roberts,  1977;  Novak,  1977;  Charniak,  1978)  than  to 
any  other  class  of  representation. 

Mathematically,  it  is  closer  to  semantic  networks,  but  our  nodes  and  links 
mean  very  different  things  (see  III.  above)  than  they  do  for  semantic  networks. 

One  difference  with  semantic  networks  is  that  our  procedures  (e.g., 
strategies,  etc.)  are  objects  of  the  same  status  as  any  other  data;  they  are 
concepts  that  are  executable.  When  procedures  are  used,  they  are  executed 
(mentally  or  physically). 

6.  Our  Approach  Contrasted  with  Paivio  and  Linguistically  Based  Theories 

The  main  difference  between  our  knowledge  representation  and  that  of 
Paivio  (1971)  and  others  is  that  in  our  representation  each  concept  contains  a 
mixture  of  elements  of  different  types:  abstract,  motoric,  visual,  and  verbal. 
(As  mentioned  in  II. A.,  Jackendorff  (1983)  hypothesizes  a  level  of  representa¬ 
tion  at  which  linguistic,  sensory,  and  motor  information  are  compatible.  He 
does  not  specify  the  structure  or  processes  in  detail,  however.)  Paivio' s 
approach  specifically  postulates  a  distinction  between  verbal  and  imaginal 
processing  (dual  code).  He  treats  both  processes  as  parallel  and  as  operating 
on  different  objects. 
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Some  other  approaches  (for  example,  Kintsch  &  van  Dijk  (1978)  and  some 
versions  of  semantic  networks)  concentrate  almost  exclusively  on  processing 
linguistic  elements,  and  put  everything  else  in  the  category  of  "world 

knowledge." 

7.  The  Role  of  Background,  or  World  Knowledge 

Our  approach  differs  from  others  on  the  role  of  background  or  world 

knowledge  in  problem  solving,  that  is,  in  doing  a  specific  task.  In  our 
framework  (as  was  briefly  mentioned  in  1 1 1. A.  above)  the  role  of  background 
seems  to  be  very  unimportant.  Namely,  either  elements  needed  to  solve  the  task 
are  already  incorporated  into  the  concept  involved  in  solving  the  task  (just  as 
some  knowledge  of  the  circuitry  is  incorporated  into  our  adult's 

conceptualization  of  a  flashlight),  in  which  case  the  concepts  used  are 

sufficient  by  themselves  to  solve  the  problem  (or  perform  the  task);  or 
necessary  elements  are  not  incorporated,  in  which  case  an  attempt  to  perform 
the  task  would  fail  until  the  concepts  are  modified  and  extended. 

IV.  Final  Remarks 

In  the  introduction  to  their  1982  book  Intel! igent  Tutoring  Systems  (ITS), 
Sleeman  and  Brown  state  that  the  designers  of  intelligent  tutoring  systems, 
such  as  Brown,  Burton,  and  deKleer  (1981),  Burton  and  Brown  (1981),  Clancey 
(1981),  Genesereth  (1981),  Goldstein  (1981),  Kimball  (1981),  Miller  (1981), 
Sleeman  and  Hendley  (1981),  and  Smith,  Graves,  Blaine,  and  Marinov  (1975),  are 
dissatisfied  with  their  system's  overall  performance.  They  continue,  "The 
following  are  some  of  the  acknowledged  shortcomings: 

(1)  The  instructional  material  produced  in  response  to  a  student's  query 
or  mistake  is  often  at  the  wrong  level  of  detail,  as  the  system  assumes  too 
much  or  too  little  student  knowledge. 

(2)  The  system  assumes  a  particular  conceptual izati on  of  the  domain, 

thereby  coercing  a  student's  performance  into  its  own  conceptual  framework. 

None  of  these  systems  can  discover,  and  work  within,  the  student's  own 
(idiosyncratic)  conceptualization  to  diagnose  his  "mind  bugs"  within  that 
framework. 

(3)  The  tutoring  and  critiquing  strategies  used  by  these  systems  are 
excessively  ad  hoc  reflecting  unprincipl ed  intuitions  about  how  to  control 
their  behavior.  Discovering  consi stent  principles  would  be  facilitated  by 
constructing  better  theories  of  learning  and  mislearning--a  task  requiring 
detailed  psychological  theories  of  knowledge  representation  and  belief 
revision. 

(4)  User  interaction  is  still  too  restrictive,  limiting  the  student’s 

expressiveness  and  thereby  limiting  the  ability  of  the  tutor's  diagnostic 

mechanisms."  (p.  3) 

Our  approach  contains  methods  for  potentially  reducing,  and  perhaps 

eliminating,  these  shortcomings.  Very  briefly,  we  consider  each  of  the  4 

points  above  and  how  our  system  will  treat  it. 


1.  Information  at  the  wrong  level  of  detail.  ( ITS) 

Our  system  (program)  will  discover  what  a  subject's  conceptualization  is, 
namely,  what  the  subject  knows.  Using  a  concept  comparison  procedure,  it  will 
compare  that  conceptualization  to  what  the  conceptualization  should  be. 
Therefore  it  can  be  very  much  on  target  in  providing  the  information  the 
subject  needs  to  know  to  properly  modify  his  concept.  (It  can  be  far  more 
accurate  than  device  model  instruction  (Kieras  and  Bovair,  1983)  as  well; 
device  model  instruction  provides  the  same  information  for  everybody. )  The 
system  can  also  test  if  the  information  provided  is  incorporated  into  the 
subject's  conceptualization. 

2.  The  system  coerces  a  student  into  its  own  conceptual  framework.  ( ITS) 

Our  approach  can  potentially  make  a  large  contribution  here.  Our  system 
(program)  will  derive  an  individual  subject's  own  conceptualization,  and 
communicate  with  the  subject  using  his  own  conceptualization. 

3.  Strategies  are  based  on  unprincipled  intuitions.  Better  theories  of 
learning  need  to  be  constructed.  ( ITS) 

We  have  (in  our  theoretical  paper,  parts  I  and  II)  the  beginnings  of  a 
detailed  theory  of  learning.  This  research  will  be  an  opportunity  to  give 
it  a  major  test. 

4.  User  interaction  limits  a  student's  expressiveness,  thereby  limiting  the 
tutor's  diagnostic  ability.  ( ITS) 


First  we  consider  subject- to- prog ram  communication.  In  our  past  work 
we  have  found  that  we  can  construct  a  subject's  conceptualization  of  the 
structure  of  an  object  he  is  building,  from  an  abstract  graph  and  order  of 
request  for  pieces.  Such  information  can  be  obtained  without  any  verbal 
input  from  the  subject.  This  approach  will  be  included  in  the  design  of  the 
concept  building  procedure  in  our  program.  It  will  rely  very  little  on  the 
linguistic  content  of  a  subject's  response,  but  mainly  on  the  order  of 
responses  and  possibly  the  time  between  responses.  Therefore,  construction 
of  the  subject's  "conceptualization,"  which  is  needed  for  a  diagnostic  of 
the  subject's  problem,  will  not  be  severely  restricted  by  limitations  of  the 
parser. 

Second,  we  consider  program-to-subject  communication.  In  our  present 
work  we  have  found  that  a  properly  organized  visual  presentation  plays  a 
major  role  in  instructions  for  how  to  build  an  object.  The  accompanying 
linguistic  information  clearly  plays  a  far  smaller  role.  The  planned 
program  is  heavily  biased  toward  presenting  pictorial  output.  (Of  course, 
the  actual  ratio  of  verbal  to  nonverbal  material  will  be  determined 
experimentally.) 

A  significant  difference  between  the  program  presented  here  and  some 
other  "successful"  programs  (for  example,  MYCIN  (Shortliffe,  1974,  1976)  and 
GUIDON  (Clancey,  1981)  is  as  follows.  In  MYCIN  and  GUIDON,  for  example, 
there  is  a  clear  division  between  the  expert  part  and  the  teaching  part. 
Here,  we  do  not  have  that  distinction  at  all.  All  elements  (student 


modeling,  factual  knowledge,  inferences  and  teaching  method)  are  treated 
uniformly. 

We  hope  that  the  theoretical  ideas  given  here,  and  their  practical 
implementation,  may  contribute  to  reducing  or  eliminating  the  main 
shortcomings  of  current  programs  that  have  been  acknowledged  by  authors  of 
intelligent  tutoring  systems. 

Researchers  working  in  cognitive  architectures  and  their  relationship 
to  behavior  (e.g.,  Anderson,  1984;  Bobrow  &  Winograd,  1977;  Langley,  1984) 
have  observed  that  for  an  intelligent  tutoring  system,  choosing  a  good 
representation  for  a  set  of  tasks  is  crucial.  A  key  point  that  we  emphasize 
is  that  our  knowledge  representation  seems  to  be  right  for  the  set  of  tasks 
that  we  plan  to  study.  For  example,  we  have  had  success  in  partially 
testing  it.  The  representation  is  simple  but  powerful  and  seems  to  contain 
the  right  components.  We  discussed  above  how  it  can  be  matched  with 
subjects'  behavior,  that  it  is  complete  and  does  not  contain  extraneous 
elements,  and  it  seems  to  be  efficient.  Further,  its  multimedia  aspect 
seems  to  be  (other  than  the  work  by  Munro  and  his  colleagues  on  the  General 
Maintenance  Training  Simulator)  unique. 

Anderson  (1984)  has  urged  that  computerized  tutors  be  used  by  many 
researchers  as  a  paradigm  for  studying  and  testing  theories  of  learning  and 
skill  acquisition.  We  hope  to  contribute  to  this  effort,  and  we  expect  that 
through  the  research  proposed  here,  our  theoretical  framework  will  develop 
into  a  full  testable  theory. 


Footnote 
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7.  batteries 


8. 

9. 

10 


one  battery 
another  battery 
and  11.  each  battery  is 
charged 

■  V_  because  the  batteries 

are  visually  identical. 


5.  front  part  with  protector 
and  bulb 

12.  protector  with  bulb 

13.  cap 

C.  snap  protector  into  back 
of  reflector 

C_1.  tilt  and  remove  protector 
with  bulb  from  back  of 
reflector. 

C_.  electrical  connection; 

2  see  29.  and  30.  below. 


12.  protector  with  bulb 


14. 

15. 

16. 


bulb 

protector 
not  burned  out 
continuous  filament  can 
be  seen  in  bulb 


D.  put  bulb  into  protector 
D_1.  take  bulb  out  of  protector 


4 

3 


1 


% 


plastic  part 
metal  part 

snap  metal  part  into 
plastic  part 

remove  metal  part 


plastic  part 

plastic  ring 
transparent  disc 

put  disc  into  ring 

remove  disc  from  ring 


metal  part 

reflector 
collar  (nondetachable) 


plastic  tube 
switch  with  contact  bar 
coil  spring  (nondetachable) 
is  seen  only  after  removing 
batteries 


switch  with  contact  bar 
switch 

contact  bar  (nondetachable) 
is  seen  only  after  removing 
batteries 


functionality 

bulb 

electric  circuit 
closed  circuit 
open  circuit 
lighted  bulb 
light  is  on 
light  is  off 

push  switch  forward 

push  switch  backward 


electrical  connection; 
see  29.  and  30.  below 
lack  of  electrical  connection 
see  29.  and  30.  below. 
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29.  closed  circuit 

30.  open  circuit 

25.  spring 
27.  bar 
22.  collar 
14.  bulb 

8.  battery 

9.  battery 

C  .  bar  is  connected 
to  collar 

C' •  bar  is  not  connected 
to  collar 

C  .  collar  is  (electrically) 
”  connected  to  bulb 
C..  bulb  is  connected  to 
battery 

C, .  battery  is  connected  to 
battery 

C  .  battery  is  connected  to 
3  spring 

C  .  spring  is  connected  to 
6  bar 

C2  is  made  by  C. 
is  made  by  A. 
and  are  made  by  B. 

CA  should  be  permanent 
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The  7  diagrams  below  (on  pages  Cl  through  C3)  represent  a  fictitious 
child’s  conceptualization  of  a  flashlight. 


1.  flashlight 

2.  structure 

3.  functionality 


2.  structure 

4.  case  with  batteries 

5.  front  part  with  protector 

and  bulb 

A.  screw  on  cap 
-1 

A  .  unscrew  cap 


4.  case  with  batteries 

6.  case 

7.  batteries 

B.  put  batteries  in 

B  take  batteries  out 
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7.  batteries 

8.  one  battery 

9.  another  battery 

10  and  11.  each  battery  is 
good 

V  *  Vg  because  the  batteries 
'  are  visually  identical 


5.  front  part  with  protector 
and  bulb 

14.  bulb 


6.  case 

23.  plastic  tube 

26.  switch 

27.  bar 

25.  coil  spring 
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3.  functionality 

L.  light  is  on 
’ .  light  is  off 

3.  push  switch  forward 

^ .  push  switch  backward 


7.  batteries  (good) 


rr'v 


Q.  r\ 

%  2, 


m  —  c  — 

€  r  jt  “ 

4  v  • 

4  >  0*.  4 
c  —  o 
true 
ec  X  4  c 
r*  »-j  T3 

“  s  i; 

•  o  e  o 

L  4  C  U 

O  x  o  a. 


n.  t  :  > 
a  r.  .*  uj 

r>  r 

i.4  *-  x  n 

C  C  X  -• 

L.  L.  i  L 

C  4  T-  C  — 

►  J  *  L  7  C 

O  C  :  »  ® 

4  ua  *•  c  k 

o  C  C  c-  & 

—  T  c  C  — 

ox  O  M  *C 


r.  4  — 

—  e  f  -  "i  i 

•r,  «r  > 


tO  O  X  • 

-»  U  O  O 
C  «  4  >*.  X 
x  A-  X  O  -« 


3  >*  x 

Z  Cl  X  U*>  3  \£> 

b  E  —  o  a:  o  3 

:  C  w  K  2 

tu  v  jj  -  — 

z  j,  < 

J  <—  >  Q.  ■ 

?  o  —  «*  *  o  x 

5  c  u  —  to 

*>  a  O  CO 

l>  C  «  —  X 

j  t  t?  n  c.  ■— •  x 

3  E  4  4.  X  -  O' 


•  a  re  re  •  u:  o 

»  4<  x  x  i.  ►—  ru 

]  Q  W  K  o  Z  — 


«  4C 

C.  c 

4  «— 

>  0  ^ 
—  -*  X  «~ 
-•COM 


Oca. 

o  x  o  «t 

O  k_  —  a. 

ft  —  O 

•  0  4  • 

C  XXX 

>  o  4  .  C  80 

X  E  4  c  u 

'  do  —  —  E  3 

■  C  ®  8C  X  x 

—  -9  4  O  fl 

X  C  <B  X 

(0  *  4  Q.  X 

I  C  L.  ®  4  — 

3  q  o  o  a. 


x  c  o. 
Co  «* 
4  -*  o-  a. 
0.^0 
4  4  • 

to  X  X  X 
O  •  C  BO 
4  4  0 
x  —  E  3 
<0  tC  x  X 
a.  a>  u  « 

OCX 

*  l.  a.  x 

4.  ffl  4  —• 
D  u  D  a. 


4  O  60 
it  C>  C  4  « 

o  <c  c  o 

•  X  c 

X  C  O  o  * 

4  l/\  O  #2  0 

c  —  X  T3  L  n: 

X  —  OC  4  4  4 

O  C  4  c»  — 

a.  D.  —  U.  O 

OX  >n 

•  >  w  •  >  c 

4  «C  (TJ  U  It  It 

o  Z  3  c  Z  CO 


0-4  4 

O  60 

1  t  3r  o 

;  4  *  —  z 

•  —  x  4  «* 
i  — •  o  x  _J 
a  e  o 
cur  t  z 
«  —  O  Ui 


4  0  0 
to  -<  >,  O' 
6C  O  <T,  © 

tt  o  a. 

CL  C 

4.  o.  ap 

«  o  o  c 

O  *s  X  u 

—  *-  c 

4  -x  ft,  * 

X  n  E  O 

<0  4  X  ft 

0-  4  L.  T 
>  »V  ^ 

•  —  a  3 

i-  C  4  O 
O  3  Q  C 


o  o.  X  r» 

•40  4 

X  X  4  — 

C  >*  u  6 

®  4  X  0  60 

>  O  3  C  c 

U>  x  X  ■* 

*  tn  « 

•  x  iT1  x  « 

o  o  so  q 

c.  n  >■  3  3 


to  tO  Cw  ft 

C  r^,  T?  > 

o  —  c  It  ■ 

3  C  X  C  Z  ‘ 

X  •»*  U.  £3  « 


>  C  —  X  tt 

C  -  •  4  —40* 

xc«>  r-  <  h-  c 

O  -  *  2  X 

-5  —  X  •  — «  C 

4  1/  C  I  It 

■  — •  J  "O  x  >  ^ 


t-  X  >. 

4  O  X  X 
3  >.  -  C 

C  r-  w 

m  o-  o  3 
4  O' 

•  4.  > 

x  c  —  -e 

c  O 

C  X  =5 

o  c 

■D  i  t  r 
o  E  i-  t- 

tS  t!  <2  <2 

®  c  r. 

•  X  t  « 
O  4  X 
O  O  CO  CO 


v;1  ’  -  .  - 

"V 

-  V  -T  -  v  - 

V  ■  \  ■  v  ■ 

i 

v-; 

I 

5 

»  /• 

<o 

«D 

ft- 

4  O 

X  c 

*J  4 

4 

o 

ra 

■  ■ . 

* 

kC 

CO 

O' 

c 

•1 

o  t?i 

G 

c 

4 

re 

>» 

X 

m 

X  -* 

-X  >» 

to 

* 

m 

H 

w  o 

4  4 

c 

•  X 

(j  P  x  /*n 

G  3  O  "*> 

4  O 

4  — 

X  4 

x-/  4 

ft-4' 

\m 

*-< 

>. 

o  o 

C  jC 

O 

C  X  o 

4  re  vc  x 

c  <C 

*n  o 

r*\  0  G 

O  *j 

C  4  fx 

4  — .  t0  4  n.  re 

>  O  m 

O  X 

G  3  >> 

•J 

c 

4 

Ob 

G 

i 

0i 

re 

4  —  >»  • 

&>  —  re  x 
£hcl  H 
X 

or 

41 

4  X 
4  U 

«  b. 

re  re 

U  L‘<  O 

c  G  L  ’  o 

;•  .j  ^ 

•j 

re  it' 

4  U1 

r  —  >  j 
:  -t  <  ff- 
3  C 

63  c  re  «t  c 

Sj  4  x-  Aj 

fx.  4  <0  f- 

4  ft'  J) 
><  G  ^  O' 

re 

X 

2  -ftJ 
rg 

re  *  rg 

4  TJ  __ _ 

u  g  r  t>  <  >» 

3  C  C  rc 

«J  4  *-x  a  4  X 

ft  t  j  < 

g  >.  in 

re  — 

C  G 

O  _  xC 

re  r  -  G 

4  o  re  '-i 
>>  g  4  *r* 
re  a 

10 

t) 

L 

c 

4-0  0  4* 

&  x.>  c 

4  X  C  ft 

c 

O  re 

a 

«->  cr 
4 

L  o  °  “■  O 

o  re  • 

—  >.  I  X  c 

4  «-*  re  «r  ■*+ 

re  >  •“ 

"*  c  >* 
a-  —  z 

O  *1  o 

re  3  •  4 

X  c  re  jc  — 

G  4  4-  4  4  0 

1  •  O  "f 

o  ftj 

ffi  >« 

g  re  *j  • 

X. 

>x 

4  O  > 

3  K 

O 

re  tj  c 

X  re  o  re  c  k 

C  ft  ft  X  ♦ 

•  re  re  o  c  re  c 

(UJ  X  4  —  4  —  X 

xX  C  ^ 

u  re  * 

Z  ->  G  x 
•  c  ^  ac 

o  -  « 
re  c  g 

4X0 

re  c  » 

c 

N 

xp  re  r 

1  u  x 

O  4  L  UJ 

re 

a 

X  c 
o  — 

Bt  C 

i>  i:  •  s  L 

c.  i  C  X  r 

P  ^  b'  X 

c  —  • 
^  —  re 

(0  4  0 

re  Jj  >  g  X  x 

o  c  re  x 

—  a:  o  0“  — J 

E  re  4 
re  re  4  — 

.6C  *^0  4. 

C 

4 

X 

G  "n  P 
m  ec 
Gm  C 

E  *J  4  •«  -  O  —> 

re  o  g  x  uj  c 

—  3  ^  ^  _  re  ^ 

—  F  ft  3 
■c  -  x.  n 
ft  ft  c  re 

4  E  4  4 

re  xj  c  o 

G  4  ft-  G 

• 

3 

•  a  ^ 
L  c  «■  c 
ODO  C 

r 

G 

•  re 

C  C'  *—  4 

3^6.  O 

f-  4  X 
—  O  G 
co  u  — 

•  mx  ro  4  •  X 

4  K  -x  .  c  ft  u. 

Q  to  m  z  O  *< 

8  4 

m  O  =>  cc 

ft 

o 

re  O'  4 

X  9  « 

xx  o  £  Oft  o 

•M-  4  L  o  —  4 

3  4  <  m  «  g 

i.  g  «r  x> 
r  ft  c  - 
U  G  to  G 

•  a  re  re 

4  ft  X  xJ 
o  G  to  G 

n 

O. 

4 

o 

c 

a 

t 

- 

§ 

X 

X 

V 

3 

4 

X 

4 

U 

c 

4 

CJ 

c 

ft 

» 

■: 

U 

C 

*) 

P 

k 

4P 

o 

to 

>* 

CJ 

(0 

C‘ 

m 

( 

\ 

4 

£ 

«0 

X 

•1 

C  4 
b-<  3 

4  c  m 

4  X  4/  C-. 
*  4  >  m 
G  4  «  <M 

c  cu 

4 

re 

G 

4 

4l 

4 

V.  t- 
O 
C  Oy 

o  n. 

X 

ec  c 
eo  o  re 
- -  r  co 

4  0  2  ^ 

N  C  O  <M 

4X2  CSJ 

re 

*o 

4 

O'  4 

O  4 

•  rv  x 

^  rg  0) 

>  rvj  re  cn 

re 

4 

4  X 
>  re 

c  xx  rg 

zo  ec  lo 

4  r  •“ 

4  C  UJ 

4 

4 

4> 

3 

a 

§ 

G 

Illinois 

61820 

1  In 

O  4 

*j  re  c 

re  in  jc  re 

4  r-  —  E  *c 

ft  ^  ft  5  rn 

X  O  4  4  *>  — 

CO  Cu  TJ  2  4  <*J 

*J  4  4  0 

X  4 

X  4 

4  yP 

4  >,  3 

4  x  c.  in 

re  —  r  c 
4  re  f  — . 
C  4  G  =S 

G 

, 

60 

60 

£ 

*4  «  ft  < 
4  *  > 
4  G  C 

O  X  • 

■—  >>  C  re 
4  E  4  — 
*->  L  r,  l. 

re 

X 

10 

4  «r 
>  > 

X  - 

b->  c 

4  O 

4  4  *C  • 

U.  yJ  < 

JO  ftO  X 

4  4 

o  re  c  c  • 

m  c  re  o  v 

>H  o  4  *->  42 

u. 

ec 

MX  — .  N 

c  «r  u.  in  re 

>  c 

C  TJ  4  C 

O  •  4  r.  4/ 

re  c  re  re  — 

—  o  x  «  > 

3  0  «r 

o  xx  G  G 
xX  XX  o 
(4  4 

*  X 

re  )  c  ec 

O  4  4  4 

3  4- 
X  0 

4 

o 

G  C 
ft 

K  —J  C 

o  *-i  (0 

4 

>s  •  G 

G  C 

—  4 

X  G  G  «.  «C 

CJ  JC  to  Z 

4  •  >4 

re  c  at  c  c 

0  4  0  re  0  4 

—  r  *j  c  4  p  <c 

O  >  G 

->  c  «t 

—  E  G 

4  O  X 
re  c  • 

X  *C  ft  TJ 

• 

4  u  c 

O  •  re 

to  —  W 

•  O  4) 

4  •  G  r-i 

03(^4 

re 

X 

4 

O 

2  60 
C 

c  -« 

rj  -h 
ur>  4 

#  P  ^  3  - 
^  re  0  4 

o  X  x 

•  =  -«  E 

4  -c  o  c«  re 

£  U  ffi  •-  ft) 

4 

G 

<C 

•  a. 
4  az 
Ct  « 

3k  4  er  o  — 

c  o  —  —  a: 

o  — <  x  c  ft- 

C  — «  •  4  —  ro 

a  4  4  —  x 

—  «  O  xJ  <  « 

^  60P  i 
■J  c  ft  re 

C  re  *J 
•  4  a  *> 

4  re  4  x 

O  u  n  D_ 

4  *-> 
6C  ft 

re 
•  o. 

4  4 
O  G 

4  re  3 

4  Cl  -3 

>  E 
—  re 

C  X  4 

G>  G  G 

f-  G  C  O  CD  3  — 

—  *6  C  4 

4  >.  X  -■»  Z  X 

TJ  >  re  •  —  E 

o  re  re  4  o  c  re 

U  Z  It  DElTU 

—  C  x  o 

Z  G  u  G 

c  re  c 
•  re  Q.  ro 

4  X  4  X 

G  to  G  tO 

| 

- 

2 

’ 

E 

« 

6 

>  ,■ 

IT> 

ft' 

t- 

_ 

o 

in 

«o 

c> 

r 

« 

G 

>» 

p 

O 

4  •> 

4  «  O 

►» 

4  x> 
re  4 

re 

o 

tt 

U 

o 

' 

>  - 

in 

(0 

m 

re 

a- 

4J  —  — 

*>  4 

4  3 

X  c  o  c 

o 

H mfe. 

H 

>» 

4  ec 

4  T?  9 
P  ^  - 

>» 

X  4 

re  >  c-» 

00 

fx 

4  4  re 

G  O  — 

rv  G  D 

b  a 

4  C 

C' 

in 

r  c 
c  re  4 

re  re  re  c 

-- _ re  4  x 

X 

re 

00 

o 

U 

K 

u 

a 

< 

E 

4 

—i  4  (\i 

C  G  •  Cy 

4- 

£ 

63  G  in  re 

c  —  4 

C  n 

o 

•o  rg  c  —  c 

<*:  c\i  re  re  — 

O'  E  G  -J 

O  4  Z 

G  *J  w 

ry  re  C 

O 

fy 

re  c  *->  9 
P  r.  l  C 
4  G  re  o 

Oft—  —  —  >■ 

t  r  c  4  c  re 

<  C  C  G’  G  G 

O-  3 

C  TJ 

UJ 

3  O 

Z  Cy 

. 

jn 

«x  C  O  G 

c^o 

*J  — 

re  *j 

re  h  — • 

— .  T  «C  UJ  ft-  4- 

O'  xx  re  ^ 

UJ  P  > 

G 

G 

X  G  ft 

>.  >  re  g  g 

OC  G 

:  H 

- 

* 

T> 

c 

X  — <  X 

cj  x  c  * 
||  R  P  ft 

re 

X 

re 

0—4  X> 

4  re  *  w 

4>  z  G  X 

*>  re 
-to  — 
P  4 

xx  C  G  O  O 

—  C  >» 

IP  O  4  >*  xJ 

«  re  re 

G  TJ  —  2 

4  re 

c 

£  cc'  Si 

_  >*  ^ 

—  X  ft 

XX  >,  TJ  >*  x-J 

c  >. 

G  X  X 

to  - 
c 

« 

4 

U. 

ep 

4  4 

g 

4  4  >•  4  UJ 

Z  '-O 

C  4  S)  G  —  4 

X  i.  1  g  re  E 

re  x  «c  x 

—  CJ  4/ 

p 

UJ 

re  —  o  a 

re  re  —  v 

U  «  T  «  — *  C 

».  r  v  re  l 

*  G  ft 

4  E 

X  X 

tn  ec 

1 

r- 

N 

re  g  o  c 

C  C  4  fC 

ft»  »-*  i  x 

o 

•  G 

C 

£ 

—  60  —  X  T 

4  4  C  re  4 

re  c  4  jj  re 

«r  re 

k  x  : 

ft-  re  u 

O  G  —  4  •* 

->  O  •>  4  4 

>»  >  re 

O  a:  «.»  g 

->  3  0 

X 

*0  4  X  4 
>  •>  4 

ft  Z  4  4 

G  <**  x  >  r. 

T  4  4  0 

x-  re  re  9 

C  X 

r-i 

» 

5  O  < 

*J  — 

G 

4 

O 

4  re  ft/  —  r* 
fiUMt 

2  re  o 

G  Z  G 

4  re  re  u  c  ft 

O  2  to  G  r>  G 

re  ft*- ft" 

— J  o  o 

re 

U 

4  c  re  re 
O  33  G  _1 

oz  c r  i  4  c  c 

ui  -j  a.  0  3  0 

C  T*  ft  to 
re  G  G  X 

r.  ro 

-  2 

L\-. 

X 

a 

t- 

4 

o 

c 

8 

* 

in 

9 

9 

• 

X 

V 

3 

1 

er 

X 

CJ 

4 

re 

4 

G 

re 

C 

4* 

4 

£ 

re 

c 

C 

♦—  O 

.  o 

c  c 

X 

ec 

4 

3 

X  C*-1 

> 

c  — 
o  *» 
—  a 
p  — 

£o 

-  r\j 

G 

4 

re 

4 

X 

(J 

4 

re 

4 

G 

«t 

c 

re  — ■ 

X  o 

Cl  X  c  . 

4  t  C 

ro  »’  »n 

4  4  « 

■a 

4-  —  C  X 
4  X  ft  O 

—  x  r  c. 

r  t  t~  - 

IT 

»—> 

r-. 

r  j 

r 

r  co  »  .  o 

\.,r. 

4  r- 

i  '  ■ 

C,  ' . 

u  re  — 

4  c  *j  *>  eg 

6  ..  C  if 

Cm  i  •  s  -  *-  X  • 

C'  ft  g  w  ore 

re  *> 

i.-*  t  — 

O  3  C 

T'  >  - 

ft*  Vi 

ft  o 

3  'ft 
;•  r* 

. 

£  o 

J  9 

■o  r 

xx  re  r~ 

■  r_ 

•  t  r. 

c  X.  tv  'ft 

m  >  — 

rg  re  - 

O  G  i 

G>  *— 

'■  V>  - 

cr  rv 

>s  “» 

—  C  'ft 

V 

U  —  G 

ft. ' 

G 

+*  v  r  r 

>.  *T 

c  ft  «C  r 

o  ft-  t  .  G  X  G 

re  g  i 

G 

— <  G 

t.  :•  >.  «  r 

O  G  C 

>  —  « 

J*. 

£ 

X  >  ^ 

jr  ft'.  X 

o  :* 

no  r.  , 

o  o  r 

1/  4  w 

r 

X  « 

•  r  G 

i  re  'j  >  3 

3  2  r  ». 

re  !>. 

jr  6 

re  G  ►« 

f  -  c 

r  c-  •  c. 

«;  r  4  .c  ft  3 

G  •>  1 
•  C  *i 

•  ■  c 

re  -o  • 

g  r»  r  — 

f-  C.  XX 

ft-  c.  a:  c 

OCX.-? 

r. 

i.  .  t*  t:  C* 

rt  tj 

r  6f  *•  a  i  i- 

c  4  r  — ) 

;  •/ 

ti  re  e 

.1C  *-  *'  «  ' 

n.  «*  X  |- 

■  4  ■ 

* 

ft  4  ft/. 

L 

ft-  •< 

G  o  .-ft 

*.  l  to  ft. 

ec 
z  r. 

> .  —  ft  i  x  «  t- 

►J  c  i  re  -j  re 

4  Z  t.  -  ■ 

r  - 

T  *“  t 

r  c  rr  r  r 

V  8  l 

v  2.  r 

—  O  ft  P 

r.  ^ 

“  1  V  . 

/  'H 

ft  >  c-  -  — J 

a  •  g  r 

>  •>. 

C>  ), 

U  4  r,  *■  x*  >• 

:l  V  • 

G  T*  C  .x 

p  *  ./ 

4  ft 

r.  ^ 

4 

a 

'ft.  i  >» 

a.  u>  ft*  o.  r  - 

<  .  V. 

u.  -t 

i  ft  t  r  ft  >c 

u  ,  i  n  f  .  r-  t  •• 

«  *  4  4* 

CJ  t)  •* 

i 

L-  i-  ; 

C-  ft  ■  '  •  M  l  » 

G  t  ••  *  •«  C> 

•  t  .1  CP  G 

o  tt  »r 

r/.-; 

• 

. 

’  0  *  •  .  • 

nA>  ,-  .  •  *  •  . 

■  "*  ft  *  Ji  ■  .  *  . 

•  .  "  •  * 

; 

-  -  -  • 

*--  mLl 

-  «  '  .'••  '*  .**■ 

>  A  . 

